DIGITAL NIST: An examination of the obstacles and opportunities in the digital transformation of NIST's reference materials.

Early in 2022, NIST embarked on a pilot project to produce digital calibration reports and digital certificates of analysis for reference materials. The goal is to produce examples of digital reports and certificates to assess the scope and challenges of digital transformation in those particular measurement services. This paper focuses on the Reference Material Certificate effort of the pilot project. Our aims for this part of the pilot project are: to generate a digital Reference Material Certificate from certification data; descriptive information about the material, and other data and metadata as needed; to generate a human-readable report from the digital Reference Material Certificate; and to hold a workshop to gather stakeholder feedback. The challenges for NIST include the diverse and complex information presently contained in NIST certificates, converting values to non-SI units to match the needs of stakeholders, and format updates to NIST Reference Material Certificates necessary to allow for machine generation. Other practical challenges include the wide variety of Reference Materials offered by NIST, as well as the needs of internal and external stakeholders. This presentation will report on the progress of the NIST effort and discuss some of the challenges and solutions to producing Digital Reference Material Certificates.

Playing catch-up in building an open research commons.

As efforts advance around the globe, the US falls behind.

Now Is the Time to Build a National Data Ecosystem for Materials Science and Chemistry Research Data.

Research organizations are critically in need of directed growth toward future interoperability and federation. The purpose of this Viewpoint is to alert the government, academia, professional societies, foundations, and industries of a further need for consideration of data in chemistry and materials as a long-term and sustained development in the US. This paper is a call for coordinated action from the government, academia, and industry to establish a national strategy and concomitant infrastructure focused on research data.

Implementing a Registry Federation for Materials Science Data Discovery.

As a result of a number of national initiatives, we are seeing rapid growth in the data important to materials science that are available over the web. Consequently, it is becoming increasingly difficult for researchers to learn what data are available and how to access them. To address this problem, the Research Data Alliance (RDA) Working Group for International Materials Science Registries (IMRR) was established to bring together materials science and information technology experts to develop an international federation of registries that can be used for global discovery of data resources for materials science. A resource registry collects high-level metadata descriptions of resources such as data repositories, archives, websites, and services that are useful for data-driven research. By making the collection searchable, it aids scientists in industry, universities, and government laboratories to discover data relevant to their research and work interests. We present the results of our successful piloting of a registry federation for materials science data discovery. In particular, we out a blueprint for creating such a federation that is capable of amassing a global view of all available materials science data, and we enumerate the requirements for the standards that make the registries interoperable within the federation. These standards include a protocol for exchanging resource descriptions and a standard metadata schema for encoding those descriptions. We summarize how we leveraged an existing standard (OAI-PMH) for metadata exchange. Finally, we review the registry software developed to realize the federation and describe the user experience.

Building Open Access to Research (OAR) Data Infrastructure at NIST.

As a National Metrology Institute (NMI), the USA National Institute of Standards and Technology (NIST) scientists, engineers and technology experts conduct research across a full spectrum of physical science domains. NIST is a non-regulatory agency within the U.S. Department of Commerce with a mission to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life. NIST research results in the production and distribution of standard reference materials, calibration services, and datasets. These are generated from a wide range of complex laboratory instrumentation, expert analyses, and calibration processes. In response to a government open data policy, and in collaboration with the broader research community, NIST has developed a federated Open Access to Research (OAR) scientific data infrastructure aligned with FAIR (Findable, Accessible, Interoperable, Reusable) data principles. Through the OAR initiatives, NIST's Material Measurement Laboratory Office of Data and Informatics (ODI) recently released a new scientific data discovery portal and public data repository. These science-oriented applications provide dissemination and public access for data from across the broad spectrum of NIST research disciplines, including chemistry, biology, materials science (such as crystallography, nanomaterials, etc.), physics, disaster resilience, cyberinfrastructure, communications, forensics, and others. NIST's public data consist of carefully curated Standard Reference Data, legacy high valued data, and new research data publications. The repository is thus evolving both in content and features as the nature of research progresses. Implementation of the OAR infrastructure is key to NIST's role in sharing high integrity reproducible research for measurement science in a rapidly changing world.

A Road Map for Translational Research on Artificial Intelligence in Medical Imaging: From the 2018 National Institutes of Health/RSNA/ACR/The Academy Workshop.

Advances in machine learning in medical imaging are occurring at a rapid pace in research laboratories both at academic institutions and in industry. Important artificial intelligence (AI) tools for diagnostic imaging include algorithms for disease detection and classification, image optimization, radiation reduction, and workflow enhancement. Although advances in foundational research are occurring rapidly, translation to routine clinical practice has been slower. In August 2018, the National Institutes of Health assembled multiple relevant stakeholders at a public meeting to discuss the current state of knowledge, infrastructure gaps, and challenges to wider implementation. The conclusions of that meeting are summarized in two publications that identify and prioritize initiatives to accelerate foundational and translational research in AI for medical imaging. This publication summarizes key priorities for translational research developed at the workshop including: (1) creating structured AI use cases, defining and highlighting clinical challenges potentially solvable by AI; (2) establishing methods to encourage data sharing for training and testing AI algorithms to promote generalizability to widespread clinical practice and mitigate unintended bias; (3) establishing tools for validation and performance monitoring of AI algorithms to facilitate regulatory approval; and (4) developing standards and common data elements for seamless integration of AI tools into existing clinical workflows. An important goal of the resulting road map is to grow an ecosystem, facilitated by professional societies, industry, and government agencies, that will allow robust collaborations between practicing clinicians and AI researchers to advance foundational and translational research relevant to medical imaging.

Improving Reproducibility in Research: The Role of Measurement Science.

We report on a workshop held 1-3 May 2018 at the National Physical Laboratory, Teddington, U.K., in which the focus was how the world's national metrology institutes might help to address the challenges of reproducibility of research.The workshop brought together experts from the measurement and wider research communities in physical sciences, data analytics, life sciences, engineering, and geological science. The workshop involved 63 participants from metrology laboratories (38), academia (16), industry (5), funding agencies (2), and publishers (2). The participants came from the U.K., the United States, Korea, France, Germany, Australia, Bosnia and Herzegovina, Canada, Turkey, and Singapore.Topics explored how good measurement practice and principles could foster confidence in research findings and how to manage the challenges of increasing volume of data in both industry and research.

How measurement science can improve confidence in research results.

The current push for rigor and reproducibility is driven by a desire for confidence in research results. Here, we suggest a framework for a systematic process, based on consensus principles of measurement science, to guide researchers and reviewers in assessing, documenting, and mitigating the sources of uncertainty in a study. All study results have associated ambiguities that are not always clarified by simply establishing reproducibility. By explicitly considering sources of uncertainty, noting aspects of the experimental system that are difficult to characterize quantitatively, and proposing alternative interpretations, the researcher provides information that enhances comparability and reproducibility.

Librarians and Scientists Partner to Address Data Management: Taking Collaboration to the Next Level.

PURPOSE: This study looks at the changing way in which the Information Services Office (ISO) at the National Institute of Standards and Technology (NIST) provides services to NIST scientific and technical staff throughout their research and publishing cycles. These services include the more traditional services of a research library as well as publishing NIST technical reports and The Journal of Research of NIST, and preserving and exhibiting scientific instruments and other artifacts. ISO has always prided itself on having a close relationship with its customers, providing a high level of service, and developing new services to stay in front of NIST researcher needs. Through a concerted, strategic effort since the late 1990s, ISO has developed and promoted relationships with its key customers through its Lab Liaison Program. DESIGN/METHODOLOGY/APPROACH: This paper discusses the relationship ISO has developed with the Office of Data and Informatics (ODI), how this relationship was forged, and how this collaboration will serve as a model for working with the other labs and programs at NIST. It will also discuss the risks and opportunities of this new collaborative service model, how ISO positioned itself to become an equal partner with ODI in the exploration of solutions to data management issues, and the benefits of the relationship from ODI's perspective. FINDINGS: A pattern of strategic changes to the services and activities offered by the Lab Liaison program has put ISO in the position to collaborate as peers with researchers at NIST. ORIGINALITY/VALUE: This study provides an overview of how ISO made strategic decisions to incorporate non-traditional services to support data management at NIST.